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ABSTRACT 

Project G A IN was designed to meet the special needs 
of the academically retarded junior high school student* This 
federally funded project has been on-qoing in Broward County 
(Florida) since January 1°66. The project was conceived of as a means 
to motivate and educate those students whose "dull normal" 
intellectual ability miqht otherwise doom them to failure. Although 
this has remained the qoal of Project GAIN, through the years various 
facets of the program have been considered less than stable; a 
thorough and competent evaluation has been very difficult to achieve, 
^he authors of the 1 Q 6Q-70 evaluation conclude from their data that 
Project GAIN was not a success this year. Further, some reasons for 
questioning the positive trends in the past have been put forth. 

These trends have usually been due to interactions between schools 
and treatm°nt (differential effects) and may well have reflected mere 
artifacts. Many associated with the project can point to individual 
pupils whom they believe have profited from the GAIN program. 

However, in^terms of outcomes for qroups of students rather than for 
individual, cases, the Droqram is considered to have little, if any, 
effect. (Authcr/JW) 
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PROJECT GAIN 



Project GAIN is a program designed to meet the special needs of 
the academically retarded junior high school student. It is a 
federally-funded project which has been ongoing in Broward County 
since January, 1966. 

ThO project was conceived of as a means to motivate and educate 
those students whose "dull normal" intellectual ability might other- 
wise doom them to failure. 

Although this has remained the goal of Project GAIN> through the 
years various facets of the program have been less than stable. As 
a result, a thorough and competent evaluation has been very difficult 
to achieve. 
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method 



Selection of S ub jects 

^ During the 1969-70 school year there were eight Junior high schools 
In Broward County meeting the criteria for ESSA Title I funds. In each 
of these schools the students who participated in 1968-69 GAIN projects 
as seventh graders were again participating as eighth graders. Unfor- 
tunately, the criteria originally used to select these students were 
rather obscure, and as a result they were dropped from this year's study. 

The sample for the 1969-70 Project CAIN was., therefore, restricted 
to the incoming seventh graders in each school. Selection of the sample 
was carried out by the Diagnostic Center of the Broward County School 
System. The criteria for selection were as follows: 

Ability — student should be within the 70 to 90 I.Q. range 
-as measured by a standardized test ("Dull Normal"). 

Achieveraent — student should be two or more grade levels be- 
low the seventh grade as measured by a grade placement score 
or a standardized achievement test. 

The Diagnostic Center provided each school with two lists of seventh 
grade students meeting these criteria. The students on one list were 
placed in GAIN classes while the other students were used as controls. 

Selecti o n and Training of Teachers 

Individual school administrators selected teachers from their own 
faculties to participate in the GAIN project. Unfortunately, no single 
set of criteria for the selection of teachers was used. 

Training for participating teachers was in the form of inservice 
seminars held after school at the County Office. At these sessions the 
Project Director as well as guest speakers . attempted to introduce the 
GAIN teachers to new and specialized techniques useful with the academi- 
cally retarded student. *• 

Treatment 



Students participating in the GAIN program attended a special two- 
period class every day. This class was In lieu of their usual English 
and social studies classes. Attention within this class was focused 
upon an individualized language arts - social sciences curriculum. The 
Intention was to use a creative and motivating approach with an emphasis 
on thematic materials and participation in activities pertaining to the 
topic being considered. To this end numerous field trips were planned 
and carried out. . 



In order to insure some measure of individual attention, class size 
was kept dewn to :jbout fifteen students. During the 1969-70 school year 
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an additional attempt to individualize instruction for the GAIN students 
vas made through the use of a series of MINI-UNITS. These v;ere simply 
short pamphlets (usually two to four pages) with a pre-test, some motiva- 
tional material, text to be studied, and a post-test. Orderly progress 
through the series of MINI-UNITS at the student's own individual rate 
was supposed to allow learning to take place without the frustration and 
failure he might otherwise have experienced in a normal class situation. 

Plan for Evaluation and Instruments Used 



In educational research the clarity of the evaluation and the appro- 
priateness of the measures used are directly correlated to the specificity 
of the objectives of the project under study. In the present case this 
leaves much to be desired. 

As happens in many programs where a multitude of goals are sought, 
few of the goals of Project GAIN are stated with any degree of objectivity. 
They simply boil down to a desire to educate and motivate the child. That 
being the case, it was decided that a simple pre-post design utilizing, 
primarily, teacher ratings and achievement test results was appropriate. 

In order to carry out our evaluation with a minimum of interference 
to the GAIN classes, all pre-measures ‘ were taken from the students' 
articulation cards, filled out at the end of their sixth school year. 

These pre-measures were: 

1. Total Reading Score - J^lltoxniA I c I Bj3£i£ Skii Is. (CTBS) , 

6th grade 

2. Total Language - IfiSl .of .Ra&te Skills (CTBS), 

6th grade 

3. Psycho/Behavioral Variables - Ratings by 6th grade teacher 

a. Intellectual Curiosity 

b. Creativity 

c. Completes Assignments 

d. Student Cooperation 

e. Behavior v 

f. Emotional Stability 

g. Social Adjustment 

h. Personal Appearance 

i. General Health 

j. Attendance 

Post-testing vas carried out during the second week of April, 1970. 

All GAIN students v?ere tested on April 7, and all Control students were 
tested on April 9. Personnel from the Research Department administered 
the tests in order to insui*e uniformity of procedures. 

\ 

At this time students took the reading section of the elementary version 
of the Californi a Achievement Test . This test was used because it was be- 
lieved that the items were of an appropriate level of difficulty for the 
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students involved in the study* A standardized test designed for seventh 
graders would have probably resulted in a more restricted range of per- 
formance. Many pupils would have simply been lumped together at the floor 
of the test. Thus we used raw scores (number of items right) on the sub- 
sections of the simpler reading test in order to increase the possibility 
of detecting differences among pupils. These subtests were: Reading 

Vocabulary, Reading Comprehension — Following Directions, Reference Skills, 
and Interpretation of Material. 

A five-question Student Survey was also administered at tiis time. 

This survey was developed locally and was designed to measure satislac tion 
with school. It was felt by GAIN administrators that the positive expe- 
riences in GAIN classes should generalize to the entire school situation. 
Thus we simply asked students to rate how well they liked several school 
subjects. Their total score represented an overall rating that should 
reflect their general attitude toward school. 

Since a major portion of the individualization attempted in the 
GAIN classes was done by the use of MINI-UNITS, it was felt that some 
measure of their use was in order. Therefore, the week following testing 
a list was compiled of each GAIN student and the number of MINI-UNITS 
they had completed. 0u v * definition of "completion" did not include a 
set level of competency. If a child went from one MINI-UNIT to the next 
in the series, the former was considered completed. 

One Important goal of GAIN was to effect a generalized improvement in 
behavior In all areas of the student’s school functioning. To measure this, 
a teacher not involved in the GAIN program (math teacher) was asked to rate 
both GAIN and Contrc] students on the same ten Psycho/Behavioral Variables 
that were on the articulation cards. To control for biasing effects, the 
names of the GAIN and Control students were randomly listed and given to 
the math teachers to rate. 

In summary, the post-measures were: 

1. Reading Tests (CAT) 

2. Student Survey 

3. Number of MINI-UNITS completed 

4. Psycho/Behavioral Variables - Ratings by 7th grade math teachers 



RESULTS 

' Explanation of Analysi s 

The statistical technique used to evaluate much of the data in this 
evaluation is called Analysis of Covariance. This technique allows us to 
equate pupils’ scores on pre-measures so that any differences that are 
found on post-measures can be attributed to the particular treatment being 
used. In this study we were interested in comparing GATN students with a 
Control group. 
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These pre-measures are called covariates and must meet certain 
statistical requirements to be valid. We attempted to run all of our 
analyses with five covariates. When any of the covariates did not meet 
the criteria, we removed it and worked with the rest. This was repeated 
as necessary until all statistical requirements were met. Therefore, 
on any chart or table all means (average scores) are adjusted for the 
maximum number of valid covariates. 

All covariates were taken from the articulation cards. The ten 
psycho-behavioral variables on these cards were subjected to a Factor 
Analysis. (A Factor Analysis attempts to find groupings of variables 
that correlat- highly within groups but not between groups.) The three 
resulting factors were used as covariates and also as post-test (cri- 
terion) scores. These factors were called: 

1. Intellectual Adjustment (Items a + b + c on page 4 ) 

2. Social Adjustment (Items d + e + f + gon page 4 ) 

3. Health (items h + i 4* j on page 4) 

Coefficient Alpha is a measure of the internal consistency of a 
test. For our three f actorally-derived pre-measures, coefficient Alpha 
ranged from 0.75 to 0.89 while for the post-measures, the range was 
from 0.64 to 0.81. We can, therefore, state that the psycho-behavioral 
and student survey measures used to evaluate Project GAIN were adequately 
reliable. 

All analyses were carried out within races; that is, no comparisons 
between races were made on any variable. 

Comparisons 

In general our analyses of the data indicated that there were no 
differences between GAIN students and Control students. 

Table 1 gives the adjusted mean scores on the eight post-measure 
variables for white and black students in both GAIN and Control groups. 

In no case were these differences statistically significant. In other 
words, after' taking initial differences into account, there were no 
systematic differences between GAIN students and Control students whether 
white or black. 

In the case of the analyses of the data for black pupils, students 
.from five schools were used. Because of the small number of white pupils 
in the program, students from only three schools could be used in the 
comparisons among whites. One school was excluded from these analyses 
because only a few students were available for a control group. On the 
other hand, all students were used in the analyses designed to explore 
the impact of the MINI-UNITS. 

Looking at differences between schools that could have been due to 
Project GAIN gave us only a slightly different picture. On one variable, 
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Health, there was a significant interaction effect (P less than .001) 
for black students. Inspection of Chart A shows us that in two of the 
five black schools, participation in GAIN did make a difference. In 
-school number thirty-four GAIN students ended the year with a lower 
score on the Health factor than the Control students. GAIN students in 
school number thirty-six, on the other hand, ended the year with a higher 
score on the Health factor than did the Control students. 

On no other variable was there a difference between GAIN students 
and Control students due to a differential impact of GAIN from school 
to school. In general, the effect of GAIN was the same in every school, 
and as noted above this effect was of insufficient magnitude to be of 
statistical significance. 

" mini-units 

Analysis of the data on MINI-UNITS was carried out by a statistical 
method called Partial Correlation. As in the Analysis of Covariance 
method discussed above, it is helpful to regard these statistical tech- 
niques of adjusting for initial differences as follows: Given a group 

of GAIN pupils with identical scores on each of the five pre-measures, 
did the students who completed more MINI-UNITS do better on any of the 
post-tests than those who completed fewer? We hold pre-tests constant 
and let the number of units completed vary with scores on the criterion 
measures. 



When initial differences were taken into account, the correlation 
between the number of MINI-UNITS completed and most other post-measure 
scores were not significant. The one exception to the above statement 
was in the case of the score on the Following Directions sub-test of the 
Reading Test among white students. The partial correlation between the 
number of MINI-UNITS completed and the Following Directions score for 
these students was 0.34 which was significant at the 0.05 level of proba- 
bility. See the appendix for a summary of the partial correlations cal- 
culated in this phase of the study. 

Discussion and Limitations 

In previous years results such as those reported in the case of 
Health have been taken as an indication that GAIN showed a "tendency" 
to be successful. It was argued that with more time to consolidate the 
program, an increase in its effectiveness would be seen. The present 
data, unfortunately, do not lend themselves to that conclusion. Part 
of the reason for this is that this year's evaluation is more rigorous 
than previous ones. Some of the findings of previous studies may have 
reflected lack of uniformity in procedures rather than "true differences" 
which had to do with the CAIN program. 

In the past, methods of collecting data were less systematic and 
Controlled. Teachers administered the tests and returned mark-sensed 
cards to £he Research Department. By administering and scoring the tests 
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Chart A 



Scores on the Health Variable 
for Negro GAIN and Control Students in Fivr Schools 




School 




io 



ourselves this year, ve probably reduced variability between schools due 
to differential data-collecting procedures. Ve were also able to sub- 
stantially reduce the amount of missing data. 

This year the selection of GAIN and Control students was carried 
out by the Diagnostic Center. Unfortunately, strictly random procedures 
were not adhered to in making these selections. On the other hand, these 
selection procedures were probably better (or at least less haphazard) 
than in preceding years. In come schools GAIN. may have been viewed as 
a dumping ground for behavior problems. 

This year, as was probably true in the past, not all schools followed 
proper procedures for replacing students in GAIN classes who had moved 
away or dropped out. In some cases Control pupils were shifted into GAIN 
classes. 

-In spite of these problems, the total impact of this year's selection 
procedures probably also contributed to reducing differences between schools 
which would be attributed to GAIN by our statistical procedures ; but could 
in. fact merely be due to a lack of uniformity in the selection of students 
and the collection of data from school to school. 

While two of our variables did prove to be statistically significant, 
they may lack true meaning and usefulness in the absence of a corroborating 
pattern of other significant differences. 

It is all well and good to say that the GAIN program has raised the 
score on the Health factor in one school and lowered it in another, but 
what does this mean? There is no similar pattern of significant differ- 
ences on other variables within the two schools affected by Health. Ve 
can as easily consider the statistically-significant findings in the area 
of Health as due to real differences between the GAIN programs in the two 
schools or as due to mere chance artifacts of no real use or meaning. 

Ve cannot as easily dismiss the findings in the area of MINI-UNITS. 

The partial correlations between the number of MINI-UNITS and variables 
other than Following Directions are not statistically significant, but 
by their magnitudes and directions indicate a consistent tendency to be 
associated with successful performance on the part of white pupils. There 
Is, however, no such trend among black students who represent the numeri- 
cally largest group of the students served by the GAIN project. Eighty-one 
percent of the pupils Included in this evaluation were black. 

Thus this very small bright spot does not negate the overall conclusion 
we are forced to reach. Our evidence indicates Project GAIN was not a 
success this year. Further, we have posited some reasons for questioning 
whatever positive trends have been found in previous evaluations. These 
trends have usually been due to interactions between schools and treatment 
(differential effects) v and may well have reflected mere artifacts. 

Undoubtedly many persons associated with this project can point to 
individual pupils whom they sincerely feel have profited greatly from the 
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GAIN program. However, when we examine the program in terms of its 
outcomes for groups of students rather than for individual cases, the’ 
program appears to have had little, if any, effect. 

Recommendations 

Future programs similar to GAIN would probably be more likely to 
succeed if the following recommendations were strictly implemented: 

1. Objectives should be clearly defined. At least some 
specific goals and procedures for achieving these goals should 
be established and then adhered to by project participants. 

Without such a common kernel, each project school and, for 
that matter, each class will represent a different version 

of the program. If each class represents a unique implementa- 
tion of the project, there can be no common outcomes attributable 
to the program. 

2. Specific criteria should be set for the selection and 
replacement of project teachers. These criteria should include 
both special competencies and experience. Teachers should be 
given incentives for remaining with the program. 

3* A strong program of inservice education should be pro- 
vided for project teachers. One goal of such a program should 
be the development of a common understanding of the program. 
Inservice training should include provisions for monitoring 
the project to insure that a common core of project-oriented 
.procedures is being properly implemented in each class. 

4. Criteria for the selection of pupils to participate in 
the program should be applied consistently. Pupils with emotional 
problems who do not meet the aptitude and achievement criteria of 
the program should not be placed in the program. For purposes of 
evaluation, the Research Department should control the selection 
of pupils to be included in assessments of the program. 
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Appendix A 

Partial Correlations Between Number of MINI -UNITS 
Complete d and Ei ght P ost-Meas ure Var iables 



\ Variables 


Negroes 


Whites 


Student Survey 


.03 


.10 


Reading Vocabulary 


.07 


.23 


Following Directions 


.00 


.34* 


Reference Skills 


.03 


.19 


Interpretation of Materials 


.03 


.24 


Intellectual Adjustment 


.07 


.06 


Social Adjustment 


.04 


.05 


Health 


.02 


.03 



^Significant at .05 level of probability 
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